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ESTIMATION OF BIT PROBABILITY OF ERROR 

USING SYNC WORD ERROR 

RATE DATA 

Dr. Charles A. Rohde 
Information Processing Division 


SUMMARY 

Assuming bit errors are independently distributed with a con- 
stant probability of error, p e , it is shown that a simple estimator 
is highly efficient for estimation of p e . The estimator is based on 
a simple function of the number of sync words containing no bit 
errors. The estimator is shown to be maximum likelihood, mini= 
mum chi-square and modified minimum chi-square when the 
quality index reported is simply the percent of frames containing 
zero errors. An approximate confidence interval for p e is obtained 
and a determination of the number of sync words to observe in 
order to obtain an approximate confidence interval of fixed length 
is indicated. The method of scoring which can be used to obtain a 
more efficient estimator of p is described. 
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ESTIMATION OF BIT PROBABILITY OF ERROR 
USING SYNC WORD ERROR 
RATE DATA 


A. Estimators of p e for grouped sync words 

0. Introduction 

This report considers the use of sync word error rate data for determining 
the data quality insofar as it is reflected by the bit probability of error p q . In 
the reporting of the number of errors in the sync word the current practice is to 
group the sync words according to the number of errors. More precisely the 
number of sync words with 0 errors, 1 error, . . . k errors and more than k 
errors are reported. Usually k = 2 but for generality we let k be any integer 
between 0 and 27. Such grouping sacrifices some information on p e and compli- 
cates the use of statistical inference procedures. 

In order to illustrate the problems and suggest possible techniques of solu- 
tion we assume that N sync words have been observed and that p e may be treated 
as constant over such a time span. If we assume that bit errors are independently 
distributed then the probability of observing X errors in a sync word follows a 
binomial distribution, i.e. 


P[X errors] - (^P* ( 1 “P e ) 27 " X 

where p e is the probability of an error in a single bit and we have used a 27 bit 
sync word for definiteness. If for each sync word the number of errors is re- 
ported then a good estimator for p e is 


27N 

l 



where X. is the number of bit errors in the i th sync word and N is the total 
number of words observed. The properties of such an estimator can be readily 
established including confidence intervals and tests of hypotheses. 

When the data are grouped it is not obvious what to use as an estimator of 
p since the total number of errors 

1 e 


1 


• • t 



X. 


is now unknown. In order to see the difficulties, let Z Qf Z v 
as follows 


, Z k+1 be defined 


Z L - number of sync words having i errors i - 0, 1, 2, • • • , k 
Z kn ~ number of sync words having more than k errors « 

Then the likelihood or probability of observing z 0 , z v • • • , z k+1 is 


fU) = 


N! 


k+i 


11 

n„: » 


II M -.)]' 1 


(1) 


i = 0 


where 


Vii P.) = (f)Pe 1 (I-Pj 27 ' 1 


0, I, * • * , k 


k 

y^l(Pe) = 1 ~ /, (pj 

i=0 


It should be clear that estimation of p e is not a simple matter. 
1. Maximum likelihood estimation of p e . 


In the rest of this report we shall discuss and describe several methods for 
estimating p e and some other associated inference problems. The method of 
maximum likelihood chooses p e to maximize f(z.), Taking logs and differenti- 
ating leads to the equation 


k+i 

L 

i=0 


[r/(pj] 

(Pe) 


z . 

1 


0 


( 2 ) 


which must be solved for p e . From the form of the y^s it is dear that some 
sort of iterative procedure is needed. In the special case k = 0 we have 


^o(Pe) 3 {1 - P e ) 27 ; 

r, (p e ) = i - (i-pj 27 s 

Z 0 = X i 


>o'(P.) “ 27 (l - p e ) 26 

7|'(p.) = 27(1 -p e ) 26 

Zj = N - x 


where x is the number of sync words having zero errors. Then Equation 2 
becomes 


so that 


- 27 x (l - p e ) 26 (N-x)27(l-pJ 

'»-* ' 27 ’ + 1 “ (l ~Pe) 27 


26 


( 1_ Po) 


0 



2. Minimum chi-square estimation of p e . 

Another method of estimation is the method of minimum chi-square which 
chooses 6 to minimize 

1 e 


-^(Pe )] 2 Jtti, z . 2 

= 2- N ^i(Pe)" N 

i=0 

Differentiating yields the equation 



k+ 1 


L 

: si n 


z t 2 y/ (p„) 
N [^i (p.)] 2 
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I'or k = 0 the equation becomes 


or 


so that 


x 2 [-27(l ~p c ) 2 ‘] 

(N-x) 2 27(l-p„) 26 

’^(~-Pc) 27 f" " 

V[7-ir-;;')”] r ’ 0 

X 

N-x 

(1-p,) 27 

[i - (i -p,,) 27 ] 


/ x \l/27 

p c = 



The general case again requires iteration and need not yield the same estimates 
as maximum likelihood even though agreement is obtained for k ~ 0. 

3. Modified minimum chi-square estimation of p e> 

The method of modified minimum chi-square is often simpler and is based 
on minimizing 



which is simply the expression for minimum X 2 with N-y . (p e ) in the denominator 
replaced by z.. Taking the derivative with respect to p o yields the equation 

„ V* ?l (p.)n' (Pe) 

2N ^ z[ = 0 

i- 0 
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to be solved for p^. If k = 0 the equation becomes 


0 


Hence 



x 


27[l-(>-P,) 27 ](l-Pc) S6 ' 




as before. Oncu again the equations require iterative techniques in general and 
need not yield the same estimates as maximum likelihood or minimum chi-square. 

Since p c cannot be solved as an explicit function of the data (the z. f s) except 
in special cases the variance of p e and confidence interval statements about p e 
cannot be easily determined. Asymptotic statements concerning p e can be made, 
however, and since N is frequently large in satellite data situations we may 
expect the results to be reliable. 

The methods of maximum likelihood, minimum chi-square and modified 
minimum chi-square applied to estimation of p e all have the same asymptotic 
properties. In particular the asymptotic variance of p e is the reciprocal of 


E 


£ 


log f(z,) 
^Po I 
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Since 


log f(z) 


k + 1 

constant + 

i=0 



Z i lQ g"/i (Pe) 


we have 


d log f (z^) 
dp e • 
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Since the are multinomial random variables we have 


E[,. .,] 


Hence 


[d log fU)l : 

L J 


Var 7 , - + [Ejj, ] 2 

COV (Zj, Zj ) + [EzJOJ 


i / i 


N >i (p,. )[*“>! (pj] ■» [ N 'i(p,)] 2 1 j 

- N>'i ( p„ ) y j ( p c ) + N* ; i (p c ) > J (p,. ) i i 


^n'(Po) y/(p.) „ 

L L TTTpT) “ 7 ( 7 ) ' Z; 

i*0 j* 0 1 ' J ' 


k+i Vy I /p fU 


i=0 u 


y^ y t * (p.)y/(p.) 
^o 1 ^ ( p o) r i ( p «) 

i ^ j 


-N>i (p e )7, (pj +N 2 (p c )] 




k + l k + 1 


y[ (p.)yj'(p.)r N < N_1 > 


i=0 j-0 

i 1\ 


k + i 2 Jsii, \yf fp 

N(N-i) £>,'(p.) 
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since 


£ >'(pJ 

i*0 



so that the asymptotic variance of p e is 


N 



( 3 ) 


Since both / and >, are polynomials in p p the above expression can easily be 
tabulated for values of p e , 


Most satellite work yields values of p p in the range 0 £p e £. .01 so tabulation 
in this range should suffice. 


B . Estimators of p ft for ungrouped sync words 

It is of some interest to compare the asymptotic variance of p e above with 
that obtained if the data were not grouped. This givos some insight into the in- 
formation lost by grouping. If the data were not grouped and Y represents the 
total number of sync word errors then Y has a binomial distribution with param 
eters 27N and p e , i.e. 

P[Y H yJ = (Tk y (i-p c ) 27N - y 

so that p e = y/ 27N and the variance of p e is 

p. ( 1 - Pc) 

27N 

Thus the ratio of asymptotic variances 



Since y/ and y. are polynomials in p e the above ratio could be easily tabulated 
for values of p e in the ranges customarily found in telemetry d ,ita. 
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Since 



is monotone increasing in k it is clear that the most information is lost when 
k = 0 (this is also intuitively obvious). For the case k s 0 we have (see (3)) 


Vi! (p, ) = 

- 27(1 -pj 26 

?o (P.) 

* (1-Po) 27 

^'(p.) = 

27(1 "Pj 26 

>1 (P.) 

= l“(l~P c ) 27 

so that 




(Pe)]' 

27 2 [(1-Pe) 26 ] 

2 27 2 

'(1-Pe) 26 ] 2 

2- r i( P e) 

i- 0 

o-p.)” 

+ 

1 ~ 

(l-Pe) 27 


= 27 2 [(1 -P C ) 26 ] 

2 F 

1 1 1 


b-(i 

-Pe)” (l"Pe) 27 J 


27 2 (l -p e ) 26 



(1- P e ) [l (1 

"Pe) 27 . 



Thus the ratio of asymptotic variances is 


f 27 2 (l- P J 26 

l Pe ( 1 _ Pe) 

27(1 ~ p a ) 26 p e 

Iji -p.)[i- (i-p.) 27 ;. 

27 

[l-(l-Pe) 27 ] 


If p e is very small (say 1.01) then 

(1-P e ) 26 = l-26p e (l : P e ) 27 = l"27 Pe 
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Hence the ratio of asymptotic variances is approximately 


27 (l - 26 p e ) p 0 
1 - (l-27pj s 1 “ 26 p <- 


which for p e = .01 is .74, for p e K .001 is .974. 


C . Estimation of p when p n is 3mall 

1 e * c 


1, The estimator and a large sample confidence interval. 


A recommendation for practical usage is clear. If p e is expected to be small 
(say <,,01) little efficiency is lost by reporting only the number X of sync word 
frames with zero errors in N observed sync words. The estimator for p e is then 


P e 


1 


ar 


(4) 


and p c may be treated as approximately normal with mean p r and variance 


(i-p.)[i-(i-p.K 

N27 2 (l -P e ) 26 


* (t-Pe)27p e 

27 2 N[l - 26 pj 

= Pe(l-Pe) 
27N [l - 26 p e ] 


It follows that an approximate 1 - a level confidence interval for p e is 


P i z /n 
*e a/2 


p , (i-Pe) ] 

27N[l-26 $'] ' 


1/2 


where 
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can be obtained from tables of the standard normal distribution. Since 
£p e (l ~ P e )/! ~ 26 p e J is a decreasing function of p e for small p e we have that 


Pe ( ) _ Pe) , . 01( . 99) 

Inn ■ m i n i l‘ w i g£jsgwa a ii i i m ii n i i ■ 4 . ii MUTm i . a— **?■*» i i mu >7 

l-26p„ ~ - 74 


.013 


so that the interval 


Pc 


± z 


a/ 2 



1/2 


is at least a 1 - a level confidence interval for p . If we desire a confidence 

1 e 

interval of length 2d then we can set 


d « 


Z rx/2 


\ 013] 1/3 

. w . 


so that 


d 2 


2 .013 

z a/2 27N 


Hence 


N 


'a/2 


.013 


d 2 27 


will yield an approximate 1 - a confidence interval of length 2d. 
2. Asymptotic variances of more accuracy. 


If asymptotic variances are desired to more accuracy we may simply ob 
serve that 


(Pe) “ (i'Pe ) 27 ' 1 


0, 1,2, • • • , k 


10 


so 


yi (p e ) = (^Po)”' 1 + ( 27_ i )Pe‘ (! -Pc) 27 ' 1 " 1 } 


(YW -1 (l-p.) a7 - i_1 [i(l"pJ-(27-i)pJ 


i) Pe (i-Po) 27 " 1 , V 

5TF13 (l ' 27pJ 



for i = 1, 2, • • * , k. For k + 1 


r„,( p.) = i-£( 2 ’)e.‘ 


i-0 


so 


T'k+l ( P e ) ~ y ' 


S( Y)p/ (l-Pa ) 27 ' 1 ( i ~ 27 P=) 

Pe ( 1_ Pe) 


i=0 


Now define 


P(Pei><) = ^(YlPe 1 (l-Pe) 2 ^ 

i=0 


k 

M(p e ;k) = r(Y)p.‘(i-P.) 2 y -Mi-»p.) 


i = 0 


V (P e ; k ) " ^(YjPe (^Pe ) 27 ' 1 ( i ~ 27 P e ) 


i = 0 
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Then for i = 0, 1, 2, 


• • • 


, k we have 


b± ( p e )] 

M Pe) 


Cl) [p , 1 (i-Pe ) 27 ' 1 ] 2 [C 1 ~ 27 Pq)] 2 /p. 
(Y) Po‘ (l “ P e ) 27_l 


SO 


Also 


Hence 


; / 



I ) pj ( 1 ~Po) 27 ~ i 

p, 2 (i-Pc r 


(* ~ 27 Pe)‘ 


k 

L 

1 = 0 


K' ( Pc )] 

MpJ 


( 1 -P.)« V(P ° ;l0 


klM 2 [ M (P e i k )]/Pe 2 ( 1 _ Pe) 


7 k« (Pe) 


[V ( Pc ). 
^ (p.) 


k + 1 
V" 1 


Pe' ( 1 _ Pe) 


1-P(p 0 ; k) 


[ M (p e = k )] 2 

v (p e : k ) + l-P( Pe ; k) 


! { v (p e ; k ) [ l “ p (p e ; k )] + [ M (p e ; k )] 2 } 


P e 2 (! "Pe) 


1-P(p e ;k) 


Thus the ratio of asymptotic variances is 


{v(p e ; k)[l-P(p e ; 

k ) 

+ 

_m(p s ; 

k ). 


27 p ( 

= ( l "Pe) 

[>■ 

■p(p.; k \ 
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3. Recursive estimation of p . 

* e 


Let 


kti 

s(z, P j = 22 


k+1 yj± Pe] 

y. (p ) 

i=0 ' 1 ' 


Z. 


i 


Then we can write 


S(2.P.) ‘ £ 


z. (i - 27) 


M(p e ; k ) 


TT^Pei 1 Pe) P c (l “P e )[l -P(p e ! k )] 


or 


S (2. Pe) 


Pe (! 




27p e ) +Z k+1 


M(p e ; k) 


Pe(l'Pe) l-Kp.ik) 


Now 


k 

^ i Z. = total number of errors in the first k groups of sync words = Z 


i=0 


k 

Z. = total number of sync words with less than k+1 errors = Z 

i=0 


„ rj * 
2 


Hence 


S(Z, P e ) 


i J 7 *_ 27o M ( P e ; M 

(l-p e )1 Z l 27P e Z 2 +Z k + 1 


[i- p (p e ; k)] 
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Also 


k + 1 


N 


r-, [>/ (Pq)] 2 

N1 (pJ N l_j y.( p e ) P 0 2 (l-P e ) 2 


{v(p c! k)[l-P(p ci k)] + [M(p c ; k' 


[l - P(p„ i k )_ 



where 


If p e ° is a trial value of p e then a better approximation to p e is p Q ‘ ‘ p e + fc p ( 


8p e ° = S(Z; P e °)/Nl(p c °) 


The process may now be repeated using p ' to get 


P 2 = P.' + 8p ' 


The process may be stopped whenever the correction term is sufficiently 
small. The practical steps may now be indicated as follows : 


(1) Compute 


N = total number of sync words observed 


nr* 

Z t = # of errors in sync words with 0, 1, 2, • • • , k errors 


4* 

Z = # of sync words with less than k + 1 errors 


z „ +1 = n - z; 


(2) Select a trial value; say p e ° = 1 - (X/N) 1/27 where X is the number of 
sync words with zero errors. 


(3) Compute l/p e ° ( 1 - p e ° ) , M(p e ° ; k )/l - P(p e ° ; k) and 


s(p e °,z) 


Pe°(l-Pe°) 


z* - 27 p° Z* + Z 


M (p.°; k) 


k+1 


,1-P(p.°, k)" 
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(4) Compute ly/l(p ti °) and 


... . s (s.^) 

(5) Compute p e ° + Sp e ° " p e ' and return to (2), replacing p e ° by p e ' . Stop 
when Sp e l is sufficiently small. 

The above iterative procedure is easily programmed for routine estimation 
problems of the type discussed in this report. 
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